Motorized full field adaptive microscope

ABSTRACT

An optical imaging system for imaging a target during a medical procedure, the imaging system involving an optical assembly having moveable zoom optics and moveable focus optics, a zoom actuator and a focus actuator for positioning the zoom and focus optics, respectively, a controller for controlling the zoom and focus actuators independently in response to received control input, a camera for capturing an image of the target from the optical assembly, the system capable of performing autofocus during a medical procedure.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This document is a continuation application claiming the benefit of, andpriority to, U.S. patent application Ser. No. 17/103,015, filed on Nov.24, 2020, entitled “MOTORIZED FULL FIELD ADAPTIVE MICROSCOPE,” U.S.patent application Ser. No. 15/570,904, filed on Oct. 31, 2017, entitled“MOTORIZED FULL FIELD ADAPTIVE MICROSCOPE,” and InternationalApplication No. PCT/CA2015/050948, filed on Sep. 24, 2015, and entitled“MOTORIZED FULL FIELD ADAPTIVE MICROSCOPE,” all of which are herebyincorporated by reference herein in their entirety.

FIELD

The present disclosure generally relates to optical imaging systems,including optical imaging systems suitable for use in image guidedmedical procedures.

BACKGROUND

Surgical microscopes are often used during surgical procedures toprovide a detailed or magnified view of the surgical site. In somecases, separate narrow field and wide field scopes may be used withinthe same surgical procedure to obtain image views with different zoomranges. Often, adjusting the zoom and focus of such a surgicalmicroscope require the user, e.g., a surgeon, to manually adjust theoptics of the microscope, which may be difficult, time-consuming andfrustrating, particularly during a surgical procedure. As well, imagecapture cameras and light sources often are separate pieces of equipmentfrom the surgical microscope, such that the specific camera and lightsource used with a given surgical microscope may be different fordifferent medical centers and even for different surgical procedureswithin the same medical center. This may result in an inconsistency inthe images obtained, which may make it difficult or impossible tocompare images between different medical centers.

SUMMARY

In some examples, the present disclosure provides an optical imagingsystem for imaging a target during a medical procedure. The systemincludes: an optical assembly including moveable zoom optics andmoveable focus optics; a zoom actuator for positioning the zoom optics;a focus actuator for positioning the focus optics; a controller forcontrolling the zoom actuator and the focus actuator in response toreceived control input; and a camera for capturing an image of thetarget from the optical assembly, wherein the zoom optics and the focusoptics are independently moveable by the controller using the zoomactuator and the focus actuator, respectively, and wherein the opticalimaging system is configured to operate at a minimum working distancefrom the target, the working distance being defined between an apertureof the optical assembly and the target.

In some examples, the present disclosure provides a processor forcontrolling the optical imaging system disclosed herein. The processoris configured to: provide a user interface to receive control input, viaan input device coupled to the processor, for controlling the zoomactuator and the focus actuator; transmit control instructions to thecontroller of the optical imaging system to adjust zoom and focus inaccordance with the control input; and receive image data from thecamera for outputting to an output device coupled to the processor.

In some examples, the present disclosure provides a system for opticalimaging during a medical procedure. The system comprises: the opticalimaging system disclosed herein; a positioning system for positioningthe optical imaging system; and a navigation system for tracking each ofthe optical imaging system and the positioning system relative to thetarget.

In some examples, the present disclosure provides a method ofautofocusing using an optical imaging system during a medical procedure,the optical imaging system including motorized focus optics and acontroller for positioning the focus optics. The method comprises:determining a working distance between an imaging target and an apertureof the optical imaging system; determining a desired position of thefocus optics based on the working distance; and positioning the focusoptics at the desired position.

BRIEF DESCRIPTIONS OF THE DRAWINGS

Reference will now be made, by way of example, to the several figures ofthe accompanying drawings which show example embodiments of the presentdisclosure, and in which:

FIG. 1 is a diagram illustrating the insertion of an access port into ahuman brain, for providing access to internal brain tissue during anexample medical procedure;

FIG. 2A is a diagram illustrating an example navigation system tosupport image guided surgery;

FIG. 2B is a diagram illustrating system components of an examplenavigation system;

FIG. 3 is a block diagram illustrating an example control and processingsystem that may be used in the example navigation systems of FIGS. 2Aand 2B;

FIG. 4A is a flow chart illustrating an example method involved in asurgical procedure that may be implemented using the example navigationsystems of FIGS. 2A and 2B;

FIG. 4B is a flow chart illustrating an example method of registering apatient for a surgical procedure as outlined in FIG. 4A;

FIG. 5 is a diagram illustrating the use of an example optical imagingsystem during a medical procedure;

FIG. 6 is a block diagram illustrating an example optical imagingsystem;

FIGS. 7 and 8 are diagrams illustrating different perspective views ofan example optical imaging system;

FIG. 9 is a flowchart illustrating an example method of autofocusingusing an example optical imaging system;

FIG. 10 is a flowchart illustrating an example method of autofocusingrelative to a medical instrument, using an example optical imagingsystem;

FIG. 11 is a diagram illustrating an example method of autofocusingrelative to a medical instrument, using an example optical imagingsystem; and

FIG. 12 is a flowchart illustrating a method of providing an opticalimaging system for imaging a target during a medical procedure.

Similar reference numerals may have been used in different figures todenote similar components.

DETAILED DESCRIPTION

The systems and methods described herein may be useful in the field ofneurosurgery, including oncological care, neurodegenerative disease,stroke, brain trauma, and orthopedic surgery. The teachings of thepresent disclosure may be applicable to other conditions or fields ofmedicine. While the present disclosure describes examples in the contextof neurosurgery, the embodiment of the present disclosure may beapplicable to other surgical procedures that may use intraoperativeoptical imaging.

Various example apparatuses or processes are below described. No exampleembodiment described herein limits any claimed embodiment and anyclaimed embodiments may cover processes or apparatuses that differ fromthose examples below described. The claimed embodiments are not limitedto apparatuses or processes having all of the features of any oneapparatus or process described below or to features common to multipleor all of the apparatuses or processes below described. It is possiblethat an apparatus or process described below is not part of any claimedembodiment.

Furthermore, numerous specific details are set forth in order to providea thorough understanding of the disclosure. However, the embodimentsdescribed herein may be practiced without these specific details. Inother instances, well-known methods, procedures, and components have notbeen described in detail so as not to obscure the embodiments describedherein.

As used herein, the terms, “comprises” and “comprising” are to beconstrued as being inclusive and open ended, and not exclusive.Specifically, when used in the specification and claims, the terms,“comprises” and “comprising” and variations thereof mean the specifiedfeatures, steps or components are included. These terms are not to beinterpreted to exclude the presence of other features, steps orcomponents.

As used herein, the term “exemplary” or “example” means “serving as anexample, instance, or illustration,” and should not be construed aspreferred or advantageous over other configurations disclosed herein.

As used herein, the terms “about”, “approximately”, and “substantially”are meant to cover variations that may exist in the upper and lowerlimits of the ranges of values, such as variations in properties,parameters, and dimensions. In one non-limiting example, the terms“about”, “approximately”, and “substantially” may be understood to meanplus or minus 10 percent or less.

Unless defined otherwise, all technical and scientific terms used hereinare intended to have the same meaning as commonly understood by one ofordinary skill in the art. Unless otherwise indicated, such as throughcontext, as used herein, the following terms are intended to have thefollowing meanings:

As used herein, the phrase “access port” refers to a cannula, conduit,sheath, port, tube, or other structure that is insertable into asubject, in order to provide access to internal tissue, organs, or otherbiological substances. In some embodiments, an access port may directlyexpose internal tissue, for example, via an opening or aperture at adistal end thereof, and/or via an opening or aperture at an intermediatelocation along a length thereof. In other embodiments, an access portmay provide indirect access, via one or more surfaces that aretransparent, or partially transparent, to one or more forms of energy orradiation, such as, but not limited to, electromagnetic waves andacoustic waves.

As used herein the phrase “intraoperative” refers to an action, process,method, event or step that occurs or is carried out during at least aportion of a medical procedure. Intraoperative, as defined herein, isnot limited to surgical procedures, and may refer to other types ofmedical procedures, such as diagnostic and therapeutic procedures.

Some embodiments of the present disclosure relate to minimally invasivemedical procedures that are performed via an access port, wherebysurgery, diagnostic imaging, therapy, or other medical procedures, e.g.,minimally invasive medical procedures, are performed based on access tointernal tissue through the access port.

In the example of a port-based surgery, a surgeon or robotic surgicalsystem may perform a surgical procedure involving tumor resection inwhich the residual tumor remaining after is minimized, while alsominimizing the trauma to the intact white and grey matter of the brain.In such procedures, trauma may occur, for example, due to contact withthe access port, stress to the brain matter, unintentional impact withsurgical devices, and/or accidental resection of healthy tissue. A keyto minimizing trauma is ensuring that the surgeon performing theprocedure has the best possible view of the surgical site of interestwithout having to spend excessive amounts of time and concentrationrepositioning tools, scopes and/or cameras during the medical procedure.

Referring to FIG. 1, this diagram illustrates the insertion of an accessport 12 into a human brain 10, for providing access to internal braintissue during a medical procedure. The access port 12 is inserted intothe human brain 10, providing access to internal brain tissue. Theaccess port 12 comprises such instruments as catheters, surgical probes,or cylindrical ports such as the NICO BRAINPATH™. Surgical tools andinstruments can be inserted within the lumen of the access port 12 inorder to perform surgical, diagnostic, or therapeutic procedures, suchas resecting tumors as necessary. In the example of a port-basedsurgery, a straight or linear access port 12 is typically guided down asulci path of the brain. Surgical instruments would then be inserteddown the access port 12. The embodiments of the present disclosure applyequally well to catheters, DBS needles, a biopsy procedure, and also tobiopsies and/or catheters in other medical procedures performed on otherparts of the body, as well as to medical procedures that do not use anaccess port. Various examples of embodiments of the present disclosureare generally suitable for use in any medical procedure using opticalimaging systems.

Referring to FIG. 2A, this diagram illustrates an exemplary navigationsystem environment 200 which may be used to support navigatedimage-guided surgery. A surgeon 201 conducts a surgery on a patient 202in an operating room (OR) environment. A medical navigation system 205comprises an equipment tower, tracking system, displays and trackedinstruments to assist the surgeon 201 during his procedure. An operator203 may also be present to operate, control, and provide assistance forthe medical navigation system 205.

Referring to FIG. 2B, this diagram illustrates an example medicalnavigation system 205 in greater detail. The disclosed optical imagingsystem may be used in the context of the medical navigation system 205.The medical navigation system 205 may include one or more displays 206,211 for displaying a video image, an equipment tower 207, and apositioning system 208, such as a mechanical arm, which may support anoptical imaging system 500 (which may include an optical scope). One ormore of the displays 206, 211 may include a touch-sensitive display forreceiving touch input. The equipment tower 207 may be mounted on aframe, e.g., a rack or cart, and may contain a power supply and acomputer or controller that may execute planning software, navigationsoftware and/or other software to manage the positioning system 208 oneor more instruments tracked by the navigation system 205. In someexamples of the medical navigation system 205, the equipment tower 207comprises a single tower configuration, operating with dual displays206, 211; however, the equipment tower 207 also comprises otherconfigurations, e.g., dual tower, single display, etc. The equipmenttower 207 further comprises a universal power supply (UPS) to providefor emergency power, in addition to a regular AC adapter power supply.

Still referring to FIG. 2B, a portion of the patient's anatomy may beheld in place by a holder. For example, as shown the patient's head andbrain may be held in place by a head holder 217. An access port 12 andassociated introducer 210 may be inserted into the head, to provideaccess to a surgical site in the head. The imaging system 500 may beused to view down the access port 12 at a sufficient magnification toallow for enhanced visibility down the access port 12. The output of theimaging system 500 may be received by one or more computers orcontrollers to generate a view that may be depicted on a visual display,e.g., one or more displays 206, 211.

Still referring to FIG. 2B, in some examples, the navigation system 205may include a tracked pointer 222. The tracked pointer 222, comprisesmarkers 212 to enable tracking by a tracking camera 213, is used toidentify point, e.g., fiducial points, on a patient. An operator,typically a nurse or the surgeon 201, uses the tracked pointer 222 toidentify the location of points on the patient 202, in order to registerthe location of selected points on the patient 202 in the navigationsystem 205. A guided robotic system with a closed loop control is usableas a proxy for human interaction. Guidance to the robotic system isprovided by any combination of input sources, such as image analysis,tracking of objects in the operating room using markers placed onvarious objects of interest, or any other suitable robotic systemguidance technique.

Still referring to FIG. 2B, fiducial markers 212 are coupled with theintroducer 210 for tracking by the tracking camera 213, which providespositional information of the introducer 210 from the navigation system205. In some examples, the fiducial markers 212 may be alternatively oradditionally coupled with the access port 12. In some examples, thetracking camera 213 comprises a 3D infrared optical tracking stereocamera similar, such as a NORTHERN DIGITAL IMAGING® (NDI) device. Insome examples, the tracking camera 213 comprises an electromagneticsystem (not shown), such as a field transmitter that may use at leastone more receiver coil disposed in relation to at least one tool that isto be tracked. A known profile of the electromagnetic field and knownposition of receiver coil(s) relative to each other may be used to inferthe location of the tracked tool(s) using the induced signals and theirphases in each of the receiver coils. Operation and examples of thistechnology is further explained in Chapter 2 of “Image-GuidedInterventions Technology and Application,” Peters, T.; Cleary, K., 2008,ISBN: 978-0-387-72856-7, incorporated herein by reference. Location dataof the positioning system 208 and/or access port 12 may be determined bythe tracking camera 213 by detection of the fiducial markers 212 placedon or otherwise in fixed relation, e.g., in rigid connection, to any ofthe positioning system 208, the access port 12, the introducer 210, thetracked pointer 222 and/or other tracked instruments. The fiducialmarker(s) 212 may be active or passive markers. A display 206, 211 mayprovide an output of the computed data of the navigation system 205. Insome examples, the output provided by the display 206, 211 may includeaxial, sagittal and coronal views of patient anatomy as part of amulti-view output.

Still referring to FIG. 2B, the active or passive fiducial markers 212may be placed on tools, e.g., the access port 12 and/or the imagingsystem 500, to be tracked, to determine the location and orientation ofthese tools using the tracking camera 213 and navigation system 205. Themarkers 212 may be captured by a stereo camera of the tracking system togive identifiable points for tracking the tools. A tracked tool may bedefined by a grouping of markers 212, which may define a rigid body tothe tracking system. This may in turn be used to determine the positionand/or orientation in 3D of a tracked tool in a virtual space. Theposition and orientation of the tracked tool in 3D may be tracked in sixdegrees of freedom, e.g., x, y, z coordinates and pitch, yaw, rollrotations, in five degrees of freedom, e.g., x, y, z, coordinate and twodegrees of free rotation, but preferably tracked in at least threedegrees of freedom, e.g., tracking the position of the tip of a tool inat least x, y, z coordinates. In typical use with navigation systems, atleast three markers 212 are provided on a tracked tool to define thetool in virtual space, however, it is known to be advantageous for fouror more markers 212 to be used.

Still referring to FIG. 2B, camera images capturing the markers 212 maybe logged and tracked, by, for example, a closed circuit television(CCTV) camera. The markers 212 may be selected to enable or assist insegmentation in the captured images. For example, infrared(IR)-reflecting markers and an IR light source from the direction of thecamera may be used. An example of such an apparatus may be trackingdevices such as the POLARIS® system available from Northern Digital Inc.In some examples, the spatial position and orientation of the trackedtool and/or the actual and desired position and orientation of thepositioning system 208 may be determined by optical detection using acamera. The optical detection may be done using an optical camera,rendering the markers 212 optically visible.

Still referring to FIG. 2B, in some examples, the markers 212, e.g.,reflectospheres, may be used in combination with a suitable trackingsystem, to determine the spatial positioning position of the trackedtools within the operating theatre. Different tools and/or targets maybe provided with respect to sets of markers 212 in differentconfigurations. Differentiation of the different tools and/or targetsand their corresponding virtual volumes may be possible based on thespecification configuration and/or orientation of the different sets ofmarkers 212 relative to one another, enabling each such tool and/ortarget to have a distinct individual identity within the navigationsystem 205. The individual identifiers may provide information to thesystem, such as information relating to the size and/or shape of thetool within the system. The identifier may also provide additionalinformation such as the tool's central point or the tool's central axis,among other information. The virtual tool may also be determinable froma database of tools stored in or provided to the navigation system 205.The markers 212 may be tracked relative to a reference point orreference object in the operating room, such as the patient 202.

Still referring to FIG. 2B, various types of markers may be used. Themarkers 212 may all be the same type or may include a combination of twoor more different types. Possible types of markers that could be usedmay include reflective markers, radiofrequency (RF) markers,electromagnetic (EM) markers, pulsed or un-pulsed light-emitting diode(LED) markers, glass markers, reflective adhesives, or reflective uniquestructures or patterns, among others. RF and EM markers may havespecific signatures for the specific tools they may be attached to.Reflective adhesives, structures and patterns, glass markers, and LEDmarkers may be detectable using optical detectors, while RF and EMmarkers may be detectable using antennas. Different marker types may beselected to suit different operating conditions. For example, using EMand RF markers may enable tracking of tools without requiring aline-of-sight from a tracking camera to the markers 212, and using anoptical tracking system may avoid additional noise from electricalemission and detection systems.

Still referring to FIG. 2B, in some examples, the markers 212 mayinclude printed or 3D designs that may be used for detection by anauxiliary camera, such as a wide-field camera (not shown) and/or theimaging system 500. Printed markers are used as a calibration pattern,for example, to provide distance information, e.g., 3D distanceinformation, to an optical detector. Printed identification markerscomprise configurations, such as concentric circles with different ringspacing and/or different types of bar codes, among other configurations.In some examples, in addition to, or in place of, using markers 212, thecontours of known objects, e.g., the side of the access port 12, arecaptured and identified by using optical imaging devices and thetracking system.

Still referring to FIG. 2B, a guide clamp 218 (or more generally aguide) for holding the access port 12 may be provided. The guide clamp218 may allow the access port 12 to be held at a fixed position andorientation while freeing up the surgeon's hands. An articulated arm 219may be provided to hold the guide clamp 218. The articulated arm 219 mayhave up to six degrees of freedom to position the guide clamp 218. Thearticulated arm 219 may be lockable to fix its position and orientation,once a desired position is achieved. The articulated arm 219 may beattached or attachable to a point based on the patient head holder 217,or another suitable point, e.g., on another patient support, such as onthe surgical bed, to ensure that when locked in place, the guide clamp218 does not move relative to the patient's head.

Still referring to FIG. 2B, in a surgical operating room (or theatre),setup of a navigation system may be relatively complicated; there may bemany pieces of equipment associated with the surgical procedure, as wellas elements of the navigation system 205. Further, setup time typicallyincreases as more equipment is added. To assist in addressing this, thenavigation system 205 may include two additional wide-field cameras toenable video overlay information. Video overlay information can then beinserted into displayed images, such as images displayed on one or moreof the displays 206, 211. The overlay information may illustrate thephysical space where accuracy of the 3D tracking system (which istypically part of the navigation system) is greater, may illustrate theavailable range of motion of the positioning system 208 and/or theimaging system 500, and/or may help to guide head and/or patientpositioning.

Still referring to FIG. 2B, the navigation system 205 may provide toolsto the neurosurgeon that may help to provide more relevant informationto the surgeon, and may assist in improving performance and accuracy ofport-based neurosurgical operations. Although described in the presentdisclosure in the context of port-based neurosurgery, e.g., for removalof brain tumors and/or for treatment of intracranial hemorrhages (ICH),the navigation system 205 may also be suitable for one or more of: brainbiopsy, functional/deep-brain stimulation, catheter/shunt placement (inthe brain or elsewhere), open craniotomies, and/orendonasal/skull-based/ear-nose-throat (ENT) procedures, among others.The same navigation system 205 may be used for carrying out any or allof these procedures, with or without modification as appropriate.

Still referring to FIG. 2B, for example, although the present disclosuremay discuss the navigation system 205 in the context of neurosurgery,the same navigation system 205 may be used to carry out a diagnosticprocedure, such as brain biopsy. A brain biopsy may involve theinsertion of a thin needle into a patient's brain for purposes ofremoving a sample of brain tissue. The brain tissue may be subsequentlyassessed by a pathologist to determine if it is cancerous, for example.Brain biopsy procedures are conducted with or without a stereotacticframe. Both types of procedures are performed using image-guidance.Frameless biopsies, in particular, are conducted using the navigationsystem 205.

Still referring to FIG. 2B, in some examples, the tracking camera 213may be part of any suitable tracking system. In some examples, thetracking camera 213, and any associated tracking system that uses thetracking camera 213, are replaced with any suitable tracking systemwhich may or may not use camera-based tracking techniques. For example,a tracking system that does not use the tracking camera 213, such as aradiofrequency tracking system, is used with the navigation system 205.

Referring to FIG. 3, this block diagram illustrates a control andprocessing system 300 usable in the medical navigation system 205, asshown in FIG. 2B, e.g., as part of the equipment tower 207. In oneexample, the control and processing system 300 comprises one or moreprocessors 302, a memory 304, a system bus 306, one or more input/outputinterfaces 308, a communications interface 310, and storage device 312.The control and processing system 300 is interfaced with other externaldevices, such as a tracking system 321, a data storage 342, and externaluser input and output devices 344, which comprise, for example, one ormore of a display, a keyboard, a mouse, sensors attached to medicalequipment, a foot pedal, a microphone, and a speaker. Data storage 342comprises any suitable data storage device, such as a local or remotecomputing device, e.g. a computer, hard drive, digital media device, orserver, having a database stored thereon. The data storage device 342stores identification data 350 for identifying one or more medicalinstruments 360 and configuration data 352 that associates customizedconfiguration parameters with one or more medical instruments 360. Thedata storage device 342 also stores preoperative image data 354 and/ormedical procedure planning data 356. Although the data storage device342 is shown as a single device, in other embodiments, the data storagedevice 342 comprises a plurality of storage devices.

Still referring to FIG. 3, the medical instruments 360 are identifiableby the control and processing unit 300. The medical instruments 360 arecoupled with, and controlled by, the control and processing unit 300, orthe medical instruments 360 are operated, or otherwise employed,independent of the control and processing unit 300. The tracking system321 may be employed to track one or more medical instruments 360 andspatially register the one or more tracked medical instruments to anintraoperative reference frame. For example, the medical instruments 360may include tracking markers such as tracking spheres that may berecognizable by the tracking camera 213. In one example, the trackingcamera 213 may be an infrared (IR) tracking camera. In another example,a sheath placed over a medical instrument 360 may be connected to andcontrolled by the control and processing unit 300. The control andprocessing unit 300 may also interface with a number of configurabledevices, and may intraoperatively reconfigure one or more of suchdevices based on configuration parameters obtained from configurationdata 352. Examples of devices 320, as shown, comprise one or moreexternal imaging devices 322, one or more illumination devices 324, thepositioning system 208, the tracking camera 213, one or more projectiondevices 328, and one or more displays 206, 211.

Still referring to FIG. 3, exemplary aspects of the disclosure can beimplemented via the processor(s) 302 and/or memory 304. For example, thefunctionalities described herein can be partially implemented viahardware logic in the processor 302 and partially using the instructionsstored in the memory 304, as one or more processing modules or engines370. Example processing modules include, but are not limited to, a userinterface engine 372, a tracking module 374, a motor controller 376, animage processing engine 378, an image registration engine 380, aprocedure planning engine 382, a navigation engine 384, and a contextanalysis module 386. While the example processing modules are shownseparately, in some examples, the processing modules 370 are stored inthe memory 304; and the processing modules 370 are collectively referredto as processing modules 370. In some examples, two or more modules 370are used together to perform a function. Although depicted as separatemodules 370, the modules 370 may comprise a unified set ofcomputer-readable instructions, e.g., stored in the memory 304, ratherthan distinct sets of instructions. The system is not intended to belimited to the components shown. One or more components of the controland processing system 300 may be provided as an external component ordevice. In one example, the navigation module 384 may be provided as anexternal navigation system that is integrated with the control andprocessing system 300.

Still referring to FIG. 3, some embodiments are implemented by using theprocessor 302 without additional instructions stored in memory 304. Someembodiments are implemented using the instructions stored in memory 304for execution by one or more general purpose microprocessors. Thus, theembodiments of the present disclosure are not limited to a specificconfiguration of hardware and/or software. In some examples, thenavigation system 205, comprising the control and processing unit 300,may provide tools to the surgeon that may help to improve theperformance of the medical procedure and/or post-operative outcomes. Inaddition to removal of brain tumours and intracranial hemorrhages (ICH),the navigation system 205 can also be applied to a brain biopsy, afunctional/deep-brain stimulation, a catheter/shunt placement procedure,open craniotomies, endonasal/skull-based/ENT, spine procedures, andother parts of the body, such as breast biopsies, liver biopsies, etc.While several examples have been provided, examples of the presentdisclosure may be applied to any suitable medical procedure.

Referring to FIG. 4A, this flow chart illustrates an example method 400of performing a port-based surgical procedure using a navigation system,such as the medical navigation system 205, as shown in FIGS. 2A and 2B.At a first block 402, the port-based surgical plan is imported. Once theplan has been imported into the navigation system at the block 402, thepatient is affixed into position using a body holding mechanism. Thehead position is also confirmed with the patient plan in the navigationsystem (block 404), which in one example may be implemented by thecomputer or controller forming part of the equipment tower 207. Next,registration of the patient is initiated (block 406). The phrase“registration” or “image registration” refers to the process oftransforming different sets of data into one coordinate system. Data mayinclude multiple photographs, data from different sensors, times,depths, or viewpoints. The process of “registration” is used in thepresent application for medical imaging in which images from differentimaging modalities are co-registered. Registration is used in order tobe able to compare or integrate the data obtained from these differentmodalities.

Still referring to FIG. 4A, numerous registration techniques areavailable and one or more of such registration techniques may be appliedto the present example. Non-limiting examples include intensity-basedmethods that compare intensity patterns in images via correlationmetrics, while feature-based methods find correspondence between imagefeatures such as points, lines, and contours. Image registration methodsmay also be classified according to the transformation models they useto relate the target image space to the reference image space. Anotherclassification can be made between single-modality and multi-modalitymethods. Single-modality methods typically register images in the samemodality acquired by the same scanner or sensor type, for example, aseries of magnetic resonance (MR) images may be co-registered, whilemulti-modality registration methods are used to register images acquiredby different scanner or sensor types, for example, in magnetic resonanceimaging (MRI) and positron emission tomography (PET). In the presentdisclosure, multi-modality registration methods may be used in medicalimaging of the head and/or brain as images of a subject are frequentlyobtained from different scanners. Examples include registration of braincomputerized tomography (CT)/MRI images or PET/CT images for tumorlocalization, registration of contrast-enhanced CT images againstnon-contrast-enhanced CT images, and registration of ultrasound and CT.

Referring to FIG. 4B, this flow chart illustrates an example methodinvolved in registration block 406, as shown in FIG. 4A, in greaterdetail. If the use of fiducial touch points 440 is contemplated, themethod involves first identifying fiducials on images (block 442), thentouching the touch points with a tracked instrument (block 444). Next,the navigation system computes the registration to reference markers(block 446). Alternately, registration can also be completed byconducting a surface scan procedure (block 450). The block 450 ispresented to show an alternative approach, but may not typically be usedwhen using a fiducial pointer. First, the face is scanned using a 3Dscanner (block 452). Next, the face surface is extracted from MR/CT data(block 454). Finally, surfaces are matched to determine registrationdata points (block 456). Upon completion of either the fiducial touchpoints 440 or surface scan 450 procedures, the data extracted iscomputed and used to confirm registration at block 408, as shown in FIG.4A.

Referring back to FIG. 4A, once registration is confirmed (block 408),the patient is draped (block 410). Typically, draping involves coveringthe patient and surrounding areas with a sterile barrier to create andmaintain a sterile field during the surgical procedure. The purpose ofdraping is to eliminate the passage of microorganisms, e.g., bacteria,between non-sterile and sterile areas. At this point, conventionalnavigation systems require that the non-sterile patient reference isreplaced with a sterile patient reference of identical geometry locationand orientation. Upon completion of draping (block 410), the patientengagement points are confirmed (block 412) and then the craniotomy isprepared and planned (block 414). Upon completion of the preparation andplanning of the craniotomy (block 414), the craniotomy is cut and a boneflap is temporarily removed from the skull to access the brain (block416). Registration data is updated with the navigation system at thispoint (block 422). Next, the engagement within craniotomy and the motionrange are confirmed (block 418). Next, the procedure advances to cuttingthe dura at the engagement points and identifying the sulcus (block420).

Still referring back to FIG. 4A, thereafter, the cannulation process isinitiated (block 424). Cannulation involves inserting a port into thebrain, typically along a sulci path as identified at 420, along atrajectory plan. Cannulation is typically an iterative process thatinvolves repeating the steps of aligning the port on engagement andsetting the planned trajectory (block 432) and then cannulating to thetarget depth (block 434) until the complete trajectory plan is executed(block 424). Once cannulation is complete, the surgeon then performsresection (block 426) to remove part of the brain and/or tumor ofinterest. The surgeon then decannulates (block 428) by removing the portand any tracking instruments from the brain. Finally, the surgeon closesthe dura and completes the craniotomy (block 430). Some aspects of FIG.4A are specific to port-based surgery, such as portions of blocks 428,420, and 434, but the appropriate portions of these blocks may beskipped or suitably modified when performing non-port based surgery.

Referring back to FIGS. 4A and 4B, when performing a surgical procedureusing a medical navigation system 205, the medical navigation system 205may acquire and maintain a reference of the location of the tools in useas well as the patient in three dimensional (3D) space. During anavigated neurosurgery, a tracked reference frame exists that is fixed,e.g., relative to the patient's skull. During the registration phase ofa navigated neurosurgery, e.g., the step 406 shown in FIGS. 4A and 4B, atransformation is calculated that maps the frame of reference ofpreoperative MRI or CT imagery to the physical space of the surgery,specifically the patient's head. This may be accomplished by thenavigation system 205 tracking locations of fiducial markers fixed tothe patient's head, relative to the static patient reference frame. Thepatient reference frame is typically rigidly attached to the headfixation device, such as a Mayfield® clamp. Registration is typicallyperformed before the sterile field has been established, e.g., via thestep 410.

Referring to FIG. 5, this diagram illustrates use of an example imagingsystem 500, described further below, in a medical procedure. Althoughthe imaging system 500 is shown as being used in the context of anavigation system environment 200, e.g., using a navigation system asdescribed above, the imaging system 500 may also be used outside of anavigation system environment, e.g., without any navigation support. Anoperator, typically a surgeon 201, may use the imaging system 500 toobserve the surgical site, e.g., to look down an access port 12. Theimaging system 500 may be attached to a positioning system 208, e.g., acontrollable and adjustable robotic arm. The position and orientation ofthe positioning system 208, imaging system 500 and/or access port may betracked using a tracking system, such as described for the navigationsystem 205. The distance d between the imaging system 500 (morespecifically, the aperture of the imaging system 500) and the viewingtarget, e.g., the surface of the surgical site, are referred as the“working distance.” The imaging system 500 is configured for use in apredefined range of working distance, e.g., in the range of about 15 cmto about 75 cm. If the imaging system 500 is mounted on the positioningsystem 208, the actual available range of working distance may bedependent on both the working distance of the imaging system 500 as wellas the workspace and kinematics of the positioning system 208.

Referring to FIG. 6, this block diagram illustrates components of anexample imaging system 500. The imaging system 500 comprises an opticalassembly 505 (also referred to as an optical train). The opticalassembly 505 comprises optics, e.g., lenses, optical fibers, etc., forfocusing and zooming on the viewing target. The optical assembly 505 mayinclude zoom optics 510 (which may include one or more zoom lenses) andfocus optics 515 (which may include one or more focus lenses). Each ofthe zoom optics 510 and focus optics 515 are independently moveablewithin the optical assembly, in order to adjust the zoom and focus,respectively. Where the zoom optics 510 and/or the focus optics 515include more than one lens, each individual lens may be independentlymoveable. The optical assembly 505 may include an aperture (not shown),which may be adjustable.

Still referring to FIG. 6, the imaging system 500 comprises a zoomactuator 520 and a focus actuator 525 for positioning the zoom optics510 and the focus optics 515, respectively. The zoom actuator 520 and/orthe focus actuator 525 comprises an electric motor, or other types ofactuators including, for example, pneumatic actuators, hydraulicactuators, shape-changing materials, e.g., piezoelectric materials orother smart materials, or engines, among other possibilities. Althoughthe term “motorized” is used in the present disclosure, understood isthat the use of this term does not limit the present disclosure to useof motors necessarily, but is intended to cover all suitable actuators,including motors. Although the zoom actuator 520 and the focus actuator525 are shown outside of the optical assembly 505, in some examples thezoom actuator 520 and the focus actuator 525 may be part of orintegrated with the optical assembly 505. The zoom actuator 520 and thefocus actuator 525 may operate independently, to control positioning ofthe zoom optics 510 and the focus optics 515, respectively. The lens(es)of the zoom optics 510 and/or the focus optics 515 may be each mountedon a linear stage, e.g., a motion system that restricts an object tomove in a single axis, which may include a linear guide and an actuator;or a conveyer system such as a conveyor belt mechanism, that is moved bythe zoom actuator 520 and/or the focus actuator 525, respectively, tocontrol positioning of the zoom optics 510 and/or the focus optics 515.In some examples, the zoom optics 510 may be mounted on a linear stagethat is driven, via a belt drive, by the zoom actuator 520, while thefocus optics 515 is geared to the focus actuator 525. The independentoperation of the zoom actuator 520 and the focus actuator 525 may enablethe zoom and focus to be adjusted independently. Thus, when an image isin focus, the zoom may be adjusted without requiring further adjustmentsto the focus optics 515 to produce a focused image.

Still referring to FIG. 6, operation of the zoom actuator 520 and thefocus actuator 525 is controlled by a controller 530, e.g., amicroprocessor, of the imaging system 500. The controller 530 receivescontrol input, e.g., from an external system, such as an externalprocessor or an input device. The control input indicates a desired zoomand/or focus; and the controller 530 may, in response, cause the zoomactuator 520 and/or focus actuator 525 to move the zoom optics 510and/or the focus optics 515, accordingly, to achieve the desired zoomand/or focus. In some examples, the zoom optics 510 and/or the focusoptics 515 may be moved or actuated without the use of the zoom actuator520 and/or the focus actuator 525. For example, the focus optics 515uses electrically-tunable lenses or other deformable material controlleddirectly by the controller 530.

Still referring to FIG. 6, by providing the controller 530, the zoomactuator 520 and the focus actuator 525 all as part of the imagingsystem 500, the imaging system 500 may enable an operator, e.g., asurgeon, to control zoom and/or focus during a medical procedure withouthaving to manually adjust the zoom and/or focus optics 510, 515. Forexample, the operator may provide control input to the controller 530verbally, e.g., via a voice recognition input system, by instructing anassistant to enter control input into an external input device, e.g.,into a user interface provided by a workstation, using a foot pedal, orby other such user interface apparatus. In some examples, the controller530 may carry out preset instructions to maintain the zoom and/or focusat preset values, e.g., to perform autofocusing, without requiringfurther control input during the medical procedure.

Still referring to FIG. 6, an external processor, e.g., a processor of aworkstation or the navigation system, in communication with thecontroller 530 may be used to provide control input to the controller530. For example, the external processor may provide a graphical userinterface via which the operator or an assistant may input instructionsto control zoom and/or focus of the imaging system 500. The controller530 may alternatively or additionally be in communication with anexternal input system, e.g., a voice recognition input system or a footpedal. The optical assembly 505 may also include one or more auxiliaryoptics 540, e.g., an adjustable aperture, which may be static ordynamic. Where the auxiliary optics 540 is dynamic, the auxiliary optics540 may be moved using an auxiliary actuator (not shown) which may becontrolled by the controller 530.

Still referring to FIG. 6, the imaging system 500 may also include acamera 535, e.g., a high-definition (HD) camera, that captures imagedata from the optical assembly. Operation of the camera may becontrolled by the controller 530. The camera 535 may also output data toan external system, e.g., an external workstation or external outputdevice, to view the captured image data. In some examples, the camera535 may output data to the controller 530, which in turn transmits thedata to an external system for viewing. By providing image data to anexternal system for viewing, the captured images may be viewed on alarger display and may be displayed together with other informationrelevant to the medical procedure, e.g., a wide-field view of thesurgical site, navigation markers, 3D images, etc. Providing the camera535 with the imaging system 500 may help to improve the consistency ofimage quality among different medical centers. Image data captured bythe camera 535 may be displayed on a display together with a wide-fieldview of the surgical site, for example in a multiple-view userinterface. The portion of the surgical site that is captured by thecamera 535 may be visually indicated in the wide-field view of thesurgical site.

Still referring to FIG. 6, the imaging system 500 comprises athree-dimensional (3D) scanner 545 or 3D camera for obtaining 3Dinformation of the viewing target. The 3D information from the 3Dscanner 545 may also be captured by the camera 535, or may be capturedby the 3D scanner 545 itself. Operation of the 3D scanner 545 may becontrolled by the controller 530, and the 3D scanner 545 may transmitdata to the controller 530. In some examples, the 3D scanner 545 mayitself transmit data to an external system, e.g., an externalworkstation. The 3D information from the 3D scanner 545 may be used togenerate a 3D image of the viewing target, e.g., a 3D image of a targettumor that is to be resected. The 3D information may also be useful inan augmented reality (AR) display provided by an external system. Forexample, an AR display, e.g., provided via AR glasses, may, usinginformation from a navigation system to register 3D information withoptical images, overlay a 3D image of a target specimen on a real-timeoptical image, e.g., an optical image captured by the camera 535.

Still referring to FIG. 6, the controller 530 may be coupled to a memory550. The memory 550 may be internal or external of the imaging system500. Data received by the controller 530, e.g., image data from thecamera 535 and/or 3D data from the 3D scanner, may be stored in thememory 550. The memory 550 may also contain instructions to enable thecontroller to operate the zoom actuator 520 and the focus actuator 525.For example, the memory 550 may store instructions to enable thecontroller to perform autofocusing, as discussed further below. Theimaging system 500 may communicate with an external system, e.g., anavigation system or a workstation, via wired or wireless communication.In some examples, the imaging system 500 may include a wirelesstransceiver (not shown) to enable wireless communication. In someexamples, the imaging system 500 may include a power source, e.g., abattery, or a connector to a power source, e.g., an AC adaptor. In someexamples, the imaging system 500 may receive power via a connection toan external system, e.g., an external workstation or processor.

Still referring to FIG. 6, in some examples, the imaging system 500 mayinclude a light source (not shown). In some examples, the light sourcemay not itself generate light but rather direct light from another lightgenerating component. For example, the light source may be an output ofa fiber optics cable connected to another light generating component,which may be part of the imaging system 500 or external to the imagingsystem 500. The light source may be mounted near the aperture of theoptical assembly, to direct light to the viewing target. Providing thelight source with the imaging system 500 may help to improve theconsistency of image quality among different medical centers. In someexamples, the power or output of the light source may be controlled bythe imaging system 500, e.g., by the controller 530, or may becontrolled by a system external to the imaging system 500, e.g., by anexternal workstation or processor, such as a processor of a navigationsystem.

Still referring to FIG. 6, in some examples, the optical assembly 505,zoom actuator 520, focus actuator 525 and camera 535 may all be housedwithin a single housing (not shown) of the imaging system. In someexamples, the controller 530, memory 550, 3D scanner 545, wirelesstransceiver, power source and/or light source may also be housed withinthe housing. In some examples, the imaging system 500 may also providemechanisms to enable manual adjusting of the zoom and/or focus optics510, 515, similarly to conventional systems. Such manual adjusting maybe enabled in addition to motorized adjusting of zoom and focus. In someexamples, such manual adjusting may be enabled in response to userselection of a “manual mode” on a user interface.

Still referring to FIG. 6, the imaging system 500 may be mountable on amoveable support structure, such as the positioning system, e.g.,robotic arm, of a navigation system, a manually operated support arm, aceiling mounted support, a moveable frame, or other such supportstructure. The imaging system 500 may be removably mounted on themoveable support structure. In some examples, the imaging system 500 mayinclude a support connector, e.g., a mechanical coupling, to enable theimaging system 500 to be quickly and easily mounted or dismounted fromthe support structure. The support connector on the imaging system 500may be configured to be suitable for connecting with a typicalcomplementary connector on the support structure, e.g., as designed fortypical end effectors. In some examples, the imaging system 500 may bemounted to the support structure together with other end effectors, ormay be mounted to the support structure via another end effector. Whenmounted, the imaging system 500 may be at a known fixed position andorientation relative to the support structure, e.g., by calibrating theposition and orientation of the imaging system 500 after mounting. Inthis way, by determining the position and orientation of the supportstructure, e.g., using a navigation system or by tracking the movementof the support structure from a known starting point, the position andorientation of the imaging system 500 may also be determined. In someexamples, the imaging system 500 may include a manual release buttonthat, when actuated, enables the imaging system 500 to be manuallypositioned, e.g., without software control by the support structure.

Still referring to FIG. 6, in some examples, where the imaging system500 is intended to be used in a navigation system environment, theimaging system 500 may include an array of trackable markers, which maybe mounted on a frame on the imaging system 500, to enable thenavigation system to track the position and orientation of the imagingsystem 500. Alternatively or additionally, the moveable supportstructure, e.g., a positioning system of the navigation system, on whichthe imaging system 500 is mounted may be tracked by the navigationsystem and the position and orientation of the imaging system 500 may bedetermined using the known position and orientation of the imagingsystem 500 relative to the moveable support structure. The trackablemarkers may include passive reflective tracking spheres, active infrared(IR) markers, active light emitting diodes (LEDs), a graphical pattern,or a combination thereof. There may be at least three trackable markersprovided on a frame to enable tracking of position and orientation. Insome examples, there may be four passive reflective tracking spherescoupled to the frame. While some specific examples of the type andnumber of trackable markers have been given, any suitable trackablemarker and configuration may be used, as appropriate. Determination ofthe position and orientation of the imaging system 500 relative to theviewing target may be performed by a processor external to the imagingsystem 500, e.g., a processor of the navigation system. Informationabout the position and orientation of the imaging system 500 may beused, together with a robotic positioning system, to maintain alignmentof the imaging system 500 with the viewing target, e.g., to view down anaccess port during port-based surgery, throughout the medical procedure.

Still referring to FIG. 6, for example, the navigation system may trackthe position and orientation of the positioning system and/or theimaging system 500 either collectively or independently. Using thisinformation as well as tracking of the access port, the navigationsystem may determine the desired joint positions for the positioningsystem so as to maneuver the imaging system 500 to the appropriateposition and orientation to maintain alignment with the viewing target,e.g., the longitudinal axes of the imaging system 500 and the accessport being aligned. This alignment may be maintained throughout themedical procedure automatically, without requiring explicit controlinput. In some examples, the operator may be able to manually move thepositioning system and/or the imaging system 500, e.g., after actuationof a manual release button. During such manual movement, the navigationsystem may continue to track the position and orientation of thepositioning system and/or the imaging system 500. After completion ofmanual movement, the navigation system may, e.g., in response to userinput, such as using a foot pedal, indicating that manual movement iscomplete, reposition and reorient the positioning system and the imagingsystem 500 to regain alignment with the access port.

Still referring to FIG. 6, the controller 530 may use information aboutthe position and orientation of the imaging system 500 to performautofocusing. For example, the controller 530 may determine the workingdistance between the imaging system 500 and the viewing target and thusdetermine the desired positioning of the focus optics 515, e.g., usingappropriate equations to calculate the appropriate positioning of thefocus optics 515 to achieve a focused image, and move the focus optics515, using the focus actuator 525, in order to bring the image intofocus. For example, the position of the viewing target may be determinedby a navigation system. The working distance may be determined by thecontroller 530 using information, e.g., received from the navigationsystem, from the positioning system or other external system, about theposition and orientation of the imaging system 500 and/or thepositioning system relative to the viewing target. In some examples, theworking distance may be determined by the controller 530 using aninfrared light (not shown) mounted on near the distal end of the imagingsystem 500.

Still referring to FIG. 6, in some examples, the controller 530 mayperform autofocusing without information about the position andorientation of the imaging system 500. For example, the controller 530may control the focus actuator 525 to move the focus optics 515 into arange of focus positions and control the camera 535 to capture imagedata at each focus position. The controller 530 may then perform imageprocessing on the captured images to determine which focus position hasthe sharpest image and determine this focus position to be the desiredposition of the focus optics 515. The controller 530 may then controlthe focus actuator 525 to move the focus optics 515 to the desiredposition. Any other autofocus routine, such as those suitable forhandheld cameras, may be implemented by the controller 530 asappropriate.

Still referring to FIG. 6, in some examples, the viewing target may bedynamically defined by the surgeon, e.g., using a user interfaceprovided by a workstation, by touching the desired target on atouch-sensitive display, by using eye or head tracking to detect a pointat which the surgeon's gaze is focused and/or by voice command, and theimaging system 500 may perform autofocusing to dynamically focus theimage on the defined viewing target. This may enable the surgeon tofocus an image on different points within a field of view, withoutchanging the field of view and without having to manually adjust thefocus of the imaging system 500.

Still referring to FIG. 6 and ahead to FIG. 11, in some examples, theimaging system 500 may be configured to perform autofocusing relative toan instrument using in the medical procedure. For example, the positionand orientation of a medical instrument, such as a tracked pointer 222,may be determined and the controller 530 may perform autofocusing tofocus the captured image on a point defined relative to the medicalinstrument. The tracked pointer 222 may have a defined focus point atthe distal tip of the pointer 222. As the tracked pointer 222 is moved,the working distance between the optical imaging system 500 and thedefined focus point (at the distal tip of the tracked pointer 222)changes (from D1 in the left image to D2 in the right image, forexample). The autofocusing may be performed similarly to that describedabove, however instead of autofocusing on a viewing target in thesurgical field, the imaging system 500 may focus on a focus point thatis defined relative to the medical instrument. The medical instrumentmay be used in the surgical field to guide the imaging system 500 toautofocus on different points in the surgical field, as below discussed.This may enable a surgeon to change the focus within a field of view,e.g., focus on a point other than at the center of the field of view,without changing the field of view and without needing to manuallyadjust the focus of the imaging system 500. Where the field of viewincludes objects at different depths, the surgeon may use the medicalinstrument, e.g., a pointer, to indicate to the imaging system 500 theobject and/or depth desired for autofocusing.

Still referring to FIG. 6 and ahead to FIG. 11, for example, thecontroller 530 may receive information about the position andorientation of a medical instrument. This position and orientationinformation may be received from an external source, e.g., from anexternal system tracking the medical instrument or from the medicalinstrument itself, or may be received from another component of theimaging system 500, e.g., an infrared sensor or a machine visioncomponent of the imaging system 500. The controller 530 may determine afocus point relative to the position and orientation of the medicalinstrument. The focus point may be predefined for a given medicalinstrument, e.g., the distal tip of a pointer, the distal end of acatheter, the distal end of an access port, the distal end of a softtissue resector, the distal end of a suction, the target of a laser, orthe distal tip of a scalpel, and may be different for different medicalinstruments. The controller 530 may use this information, together withinformation about the known position and orientation of the imagingsystem 500, e.g., determined as discussed above, in order to determinethe desired position of the focus optics 515 to achieve an image focusedon the focus point defined relative to the medical instrument.

Still referring to FIG. 6 and back to FIG. 2B, in examples where theimaging system 500 is used with a navigation system 205, the positionand orientation of a medical instrument, e.g., a tracked pointer 222 ora tracked port 210, may be tracked and determined by the navigationsystem 205. The controller 530 of the imaging system 500 mayautomatically autofocus the imaging system 500 to a predetermined pointrelative to the tracked medical instrument, e.g., autofocus on the tipof the tracked pointer 222 or on the distal end of the access port 210.Autofocusing may be performed relative to other medical instruments andother tools that may be used in the medical procedure. In some examples,the imaging system 500 may be configured to perform autofocusingrelative to a medical instrument only when it is determined that thefocus point relative to the medical instrument is within the field ofview of the imaging system 500. This may avoid an unintentional changeof focus when a medical instrument is moved in the vicinity of butoutside the field of view of the imaging system 500. In examples wherethe imaging system 500 is mounted on a moveable support system, e.g., arobotic arm, if the focus point of the medical instrument is outside ofthe current field of view of the imaging system 500, the moveablesupport system may position and orient the imaging system 500 to bringthe focus point of the medical instrument within the field of view ofthe imaging system 500, in response to input, e.g., in response to usercommand via a user interface or voice input, or via activation of a footpedal.

Still referring to FIG. 6 and back to FIG. 2B, the imaging system 500may be configured to implement a small time lag before performingautofocus relative to a medical instrument, in order to avoiderroneously changing focus while the focus point of the medicalinstrument is brought into and out of the field of view. For example,the imaging system 500 may be configured to autofocus on the focus pointonly after it has been substantially stationary for a predeterminedlength of time, e.g., 0.5 second to 1 second. In some examples, theimaging system 500 may also be configured to performing zooming with thefocus point as the zoom center. For example, while a focus point is inthe field of view or after autofocusing on a certain point in the fieldof view, the user may provide command input, e.g., via a user interface,voice input or activation of a foot pedal, to instruct the imagingsystem 500 to zoom in on the focus point. The controller 530 may thenposition the zoom optics 520 accordingly to zoom in on the focus point.Where appropriate, the positioning system (if the imaging system 500 ismounted on a positioning system) may automatically reposition theimaging system 500 as needed to center the zoomed in view on the focuspoint.

Still referring to FIG. 6 and back to FIG. 2B, in some examples, theimaging system 500 may automatically change between different autofocusmodes. For example, if the current field of view does not include anyfocus point defined by a medical instrument, the controller 530 mayperform autofocus based on a preset criteria, e.g., to obtain thesharpest image or to focus on the center of the field of view. When afocus point defined by a medical instrument is brought into the field ofview, the controller 530 may automatically switch mode to autofocus onthe focus point. In some examples, the imaging system 500 may changebetween different autofocus modes in response to user input, e.g., inresponse to user command via a user interface, voice input, oractivation of a foot pedal. In various examples of autofocusing, whetheror not relative to a medical instrument, the imaging system 500 may beconfigured to maintain the focus as the zoom is adjusted.

Still referring to FIG. 6 and back to FIG. 2B, in some examples, theimaging system 500 may generate a depth map. This may be performed bycapturing images of the same field of view, but with the imaging system500 focused on points at different depths to simulate 3D depthperception. For example, the imaging system 500 may automaticallyperform autofocusing through a predefined depth range, e.g., through adepth of about 1 cm, and capturing focused images at different depths,e.g., at increments of 1 mm, through the depth range. The imagescaptured at different depths may be transmitted to an external system,e.g., an image viewing workstation, where they may be aggregated into aset of depth images to form a depth map for the same field of view. Thedepth map may provide focused views of the field of view, at differentdepths, and may include contours, color-coding and/or other indicatorsof different depths. The external system may provide a user interfacethat allows a user to navigate through the depth map. In some examples,the optical imaging system 500 could be configured with a relativelylarge depth of field. The 3D scanner 545 may be used to create a depthmap of the viewed area, and the depth map may be registered to the imagecaptured by the camera 535. Image processing may be performed, e.g.,using the controller 530 or an external processor, to generate a pseudo3D image, for example by visually encoding, e.g., using color,artificial blurring or other visual symbols, different parts of thecaptured image according to the depth information from the 3D scanner545.

Referring to FIGS. 7 and 8, these diagrams illustrate perspective viewsof an example embodiment of the imaging system 500. In this example, theimaging system 500 is shown mounted to the positioning system 208, e.g.,a robotic arm, of a navigation system. The imaging system 500 is shownwith a housing 555 that encloses the zoom and focus optics, the zoom andfocus actuators, the camera, the controller and the 3D scanner. Thehousing is provided with a frame 560 on which trackable markers may bemounted, to enable tracking by the navigation system. The imaging system500 communicates with the navigation system via a cable 565 (showncutoff). The distal end of the imaging system 500 is provided with lightsources 570. The example shows four broad spectrum LEDs, however more orless light sources may be used, of any suitable type. Although the lightsources 570 are shown provided surrounding the aperture 553 of theimaging system 500, in other examples the light source(s) 570 may belocated elsewhere on the imaging system 500. The distal end of theimaging system 500 may also include openings 575 for the cameras of theintegrated 3D scanner. A support connector 580 for mounting the imagingsystem 500 to the positioning system 208 is also shown, as well as theframe 560 for mounting trackable markers.

Referring to FIG. 9, this flowchart illustrates an example method 900 ofautofocusing during a medical procedure. The method 900 may be performedusing an example optical imaging system, as disclosed herein. At step905, the position and orientation of the imaging system are determined.This may be done by tracking the imaging system, by performingcalibration, or by tracking the positioning system on which the imagingsystem is mounted, for example. At step 910, the working distancebetween the imaging system and the imaging target is determined. Forexample, the position of the imaging target may be determined by anavigation system, and this information may be used together with theposition and orientation information of the imaging system to determinethe working distance. At step 915, the desired position of the focusoptics is determined, in order to achieve a focused image. At step 920,the focus actuator is controlled, e.g., by a controller of the imagingsystem, to position the focus optics at the desired position. A focusedimage may then be captured, for example using a camera of the opticalimaging system.

Referring to FIG. 10, this flowchart illustrates an example method 1000of autofocusing relative to a medical instrument during a medicalprocedure. The method 1000 may be performed using an example opticalimaging system as disclosed herein. The method 1000 may be similar tothe method 900. At step 1005, the position and orientation of theimaging system is determined. This may be done by tracking the imagingsystem, by performing calibration, or by tracking the positioning systemon which the imaging system is mounted, for example. At step 1010, theposition and orientation of the medical instrument is determined. Thismay be done by tracking the medical instrument, e.g., using a navigationsystem, by sensing the medical instrument, e.g., using an infrared ormachine vision component of the imaging system, or by any other suitablemethods. At step 1015, the focus point is determined relative to themedical instrument. Determining the focus point may include looking uppreset definitions, e.g., stored in a database, of focus points fordifferent medical instruments, and calculating the focus point for theparticular medical instrument being used. At step 1020, the workingdistance between the imaging system and the focus point is determined.At step 1025, the desired position of the focus optics is determined, inorder to achieve a focused image. At step 1030, the focus actuator iscontrolled, e.g., by a controller of the imaging system, to position thefocus optics at the desired position. A focused image may then becaptured, for example using a camera of the optical imaging system.

Referring back to FIGS. 9 and 10, the methods 900, 1000 may be entirelyperformed by the controller of the imaging system, or may be partlyperformed by the controller and partly performed by an external system.For example, one or more of: determining the position/orientation of theimaging system, determining the position/orientation of the imagingtarget or medical instrument, determining the working distance, ordetermining the desired position of the focus optics may be performed byone or more external systems. The controller of the imaging system maysimply receive commands, from the external system(s) to position thefocus optics at the desired position, or the controller of the imagingsystem may determine the desired position of the focus optics afterreceiving the calculated working distance from the external system(s).

Referring to FIG. 12, a method M12 of providing an optical imagingsystem for imaging a target during a medical procedure, in accordancewith an embodiment of the present disclosure. The method M12 comprises:providing an optical assembly, providing the optical assembly comprisingproviding moveable zoom optics and providing moveable focus optics andhaving an aperture, as indicated by block 1201; providing a zoomactuator for positioning the zoom optics, as indicated by block 1202;providing a focus actuator for positioning the focus optics, asindicated by block 1203; providing a controller for controlling the zoomactuator and the focus actuator in response to received control input,as indicated by block 1204; and providing a camera for capturing animage of the target from the optical assembly, as indicated by block1205, wherein providing the zoom optics providing the focus opticscomprises respectively providing each of the zoom optics and the focusoptics with a plurality of independently moveable lenses operablycoupled with a linear stage and a conveyer system operably coupled withthe zoom actuator and the focus actuator for respectively controllingpositioning the zoom optics and the focus optics, the linear stagecomprising a linear guide and a guide actuator, and the conveyer systemcomprising a conveyor belt mechanism, and the conveyer system operablycoupled with the zoom actuator and the focus actuator for respectivelycontrolling positioning the zoom optics and the focus optics; andconfiguring the optical imaging system to automatically autofocus to apredetermined point relative to a medical instrument only after a timelag, whereby erroneous focus change is avoided, as indicated by block1206.

While some embodiments or aspects of the present disclosure may beimplemented in fully functioning computers and computer systems, otherembodiments or aspects may be capable of being distributed as acomputing product in a variety of forms and may be capable of beingapplied regardless of the particular type of machine or computerreadable media used to actually effect the distribution. At least someaspects disclosed may be embodied, at least in part, in software. Thatis, some disclosed techniques and methods may be carried out in acomputer system or other data processing system in response to itsprocessor, such as a microprocessor, executing sequences of instructionscontained in a memory, such as ROM, volatile RAM, non-volatile memory,cache or a remote storage device.

A computer readable storage medium may be used to store software anddata which when executed by a data processing system causes the systemto perform various methods or techniques of the present disclosure. Theexecutable software and data may be stored in various places includingfor example ROM, volatile RAM, non-volatile memory and/or cache.Portions of this software and/or data may be stored in any one of thesestorage devices. Examples of computer-readable storage media mayinclude, but are not limited to, recordable and non-recordable typemedia such as volatile and non-volatile memory devices, read only memory(ROM), random access memory (RAM), flash memory devices, floppy andother removable disks, magnetic disk storage media, optical storagemedia, e.g., compact discs (CDs), digital versatile disks (DVDs), etc.,among others. The instructions can be embodied in digital and analogcommunication links for electrical, optical, acoustical or other formsof propagated signals, such as carrier waves, infrared signals, digitalsignals, and the like. The storage medium may be the internet cloud, ora computer readable storage medium such as a disc.

Furthermore, at least some of the methods described herein may becapable of being distributed in a computer program product comprising acomputer readable medium that bears computer usable instructions forexecution by one or more processors, to perform aspects of the methodsdescribed. The medium may be provided in various forms such as, but notlimited to, one or more diskettes, compact disks, tapes, chips, USBkeys, external hard drives, wire-line transmissions, satellitetransmissions, internet transmissions or downloads, magnetic andelectronic storage media, digital and analog signals, and the like. Thecomputer useable instructions may also be in various forms, includingcompiled and non-compiled code.

At least some of the elements of the systems described herein may beimplemented by software, or a combination of software and hardware.Elements of the system that are implemented via software may be writtenin a high-level procedural language such as object oriented programmingor a scripting language. Accordingly, the program code may be written inC, C++, J++, or any other suitable programming language and may comprisemodules or classes, as is known to those skilled in object orientedprogramming. At least some of the elements of the system that areimplemented via software may be written in assembly language, machinelanguage or firmware as needed. In either case, the program code can bestored on storage media or on a computer readable medium that isreadable by a general or special purpose programmable computing devicehaving a processor, an operating system and the associated hardware andsoftware that is necessary to implement the functionality of at leastone of the embodiments described herein. The program code, when read bythe computing device, configures the computing device to operate in anew, specific and predefined manner in order to perform at least one ofthe methods described herein.

While the teachings described herein are in conjunction with variousembodiments for illustrative purposes, it is not intended that theteachings be limited to such embodiments. On the contrary, the teachingsdescribed and illustrated herein encompass various alternatives,modifications, and equivalents, without departing from the describedembodiments, the general scope of which is defined in the appendedclaims. Except to the extent necessary or inherent in the processesthemselves, no particular order to steps or stages of methods orprocesses described in this disclosure is intended or implied. In manycases, the order of process steps may be varied without changing thepurpose, effect, or import of the methods described.

What is claimed:
 1. An intraoperative optical imaging system for imaginga target during a medical procedure, the intraoperative optical imagingsystem comprising: an optical assembly comprising at least one moveablezoom optic and at least one moveable focus optic and having an aperture;a zoom actuator for positioning the at least one moveable zoom optic; afocus actuator for positioning the at least one moveable focus optic; acontroller for independently controlling the at least one moveable zoomoptic and the at least one moveable focus optic by respectively usingthe zoom actuator and the focus actuator in response to received controlinput; and a camera for capturing an image of the target from theoptical assembly, each of the at least one moveable zoom optic and theat least one moveable focus optic comprising at least one independentlymoveable lens operably coupled with a linear stage and a conveyer systemoperably coupled with the zoom actuator and the focus actuator forrespectively controlling positioning the least one moveable zoom opticand the at least one moveable focus optic, the linear stage comprising alinear guide and a guide actuator, and the conveyer system comprising aconveyor belt mechanism, and the conveyer system operably coupled withthe zoom actuator and the focus actuator for respectively controllingpositioning the at least one moveable zoom optic and the at least onemoveable focus optic, at least one of the zoom actuator and the focusactuator comprising at least one of a shape-changing material and asmart material, and the optical imaging system configured toautomatically autofocus to a predetermined point relative to a medicalinstrument only after a time lag, whereby erroneous focus change isavoided.
 2. The optical imaging system of claim 1, wherein the apertureis adjustable, wherein the optical imaging system is further configuredto: operate at a minimum working distance from the target, the minimumworking distance defined between the aperture of the optical assemblyand the target; automatically change between different autofocus modes;autofocus through a predefined depth range; capture focused images atdifferent depths through the predefined depth range; mount in relationto a moveable support structure; and one of automatically autofocus to apredetermined point relative to the medical instrument and automaticallyautofocus to a predetermined point relative to the medical instrumentonly if a focus point, relative to the medical instrument, is within afield of view of the optical imaging system, whereby unintentional focuschange is avoided, and wherein the controller is configured to:determine the minimum working distance by receiving information from anavigation system external to the optical imaging system, theinformation comprising information relating to a position and anorientation of a positioning system and information relating to aposition and an orientation of the medical instrument by using a machinevision component; and determine a desired position of the focus opticsbased on the minimum working distance, and control the focus actuator toposition the focus optics at the desired position.
 3. The opticalimaging system of claim 2, wherein the optical imaging system furthercomprises a support connector configured to removably mount the opticalimaging system in relation to the moveable support structure.
 4. Theoptical imaging system of claim 2, wherein the moveable supportstructure comprises at least one of: a robotic arm, a manually operatedsupport arm, and a moveable support frame.
 5. The optical imaging systemof claim 2, further comprising a manual release button configured toenable manually positioning the optical imaging system.
 6. The opticalimaging system of claim 1, further comprising a single housingconfigured to accommodate at least one of: the optical assembly, thezoom actuator, the focus actuator, and the camera, wherein the housingis further configured to accommodate the controller.
 7. The opticalimaging system of claim 1, wherein the controller is configured tocommunicate with a processor, the controller responsive to control inputreceived from the processor via a user interface.
 8. The optical imagingsystem of claim 1, wherein the controller is configured to communicatewith a processor, the controller responsive to control input receivedfrom the processor via an input system.
 9. The optical imaging system ofclaim 1, further comprising a three-dimensional (3D) camera configuredto capture 3D information of the target, wherein the 3D camera isconfigured to automatically focus through a predefined depth range andautomatically capture a plurality of focused images corresponding to aplurality of depths through the depth range, wherein the imaging systemis configured to generate a depth map based on the plurality of focusedimages corresponding to the plurality of depths through the depth range,wherein the depth map provides focused views of the field of view,corresponding to the plurality of depths through the depth range, andwherein the depth map comprises at least one of contours, color-coding,and any other indicator of each depth of the plurality of depths. 10.The optical imaging system of claim 1, further comprising at least onelinear stage mechanism configured to move at least one of the zoomoptics and focus optics.
 11. The optical imaging system of claim 1,further comprising at least one of: a power source, a power connectorconfigured to couple with the power source, and a light sourceconfigured to couple with the power source.
 12. The optical imagingsystem of claim 1, further comprising an array of trackable markersconfigured to track position and orientation of the optical imagingsystem by the navigation system.
 13. The optical imaging system of claim2, wherein the minimum working distance comprises a range ofapproximately 15 cm to approximately 75 cm.
 14. The optical imagingsystem of claim 1, wherein the information received from the navigationsystem comprises information relating to the position of the opticalassembly relative to a position of the target.
 15. The optical imagingsystem of claim 2, wherein the optical imaging system is supported by apositioning system, wherein the information received from the navigationsystem comprises information relating to the position of the positioningsystem relative to a position of the target, and wherein the minimumworking distance is determined by using a known position of the opticalassembly relative to the position of the positioning system.
 16. Theoptical imaging system of claim 2, wherein the minimum working distanceis determined as a distance between the aperture of the optical assemblyand the target, the target being the focus point defined relative to themedical instrument having a known position and a known orientation, andwherein the information received from the navigation system comprisesinformation relating to the known position and the known orientation ofthe medical instrument.
 17. The optical imaging system of claim 1,further comprising at least one of: a memory coupled with thecontroller, the memory configured to store image data captured by thecamera; and a wireless transceiver configured to transmit data from theoptical imaging system wherein at least one of the zoom actuator and thefocus actuator further comprises at least one of an electric motor, anengine, a pneumatic actuator, a hydraulic actuator, and a piezoelectricmaterial.
 18. A method of providing an intraoperative optical imagingsystem for imaging a target during a medical procedure, the methodcomprising: providing an optical assembly, providing the opticalassembly comprising providing at least one moveable zoom optic andproviding at least one moveable focus optic and having an aperture;providing a zoom actuator for positioning the at least one moveable zoomoptic; providing a focus actuator for positioning the at least onemoveable focus optic; providing a controller for independentlycontrolling the at least one moveable zoom optic and the at least onemoveable focus optic by respectively using the zoom actuator and thefocus actuator in response to received control input; and providing acamera for capturing an image of the target from the optical assembly,wherein providing the zoom optics providing the focus optics comprisesrespectively providing each of the at least one moveable zoom optic andthe at least one focus optic with at least one independently moveablelens operably coupled with a linear stage and a conveyer system operablycoupled with the zoom actuator and the focus actuator for respectivelycontrolling positioning the at least one moveable zoom optic and the atleast one focus optic, the linear stage comprising a linear guide and aguide actuator, and the conveyer system comprising a conveyor beltmechanism, and the conveyer system operably coupled with the zoomactuator and the focus actuator for respectively controlling positioningthe at least one moveable zoom optic and the at least one focus optic,and wherein providing at least one of the zoom actuator and the focusactuator comprises providing at least one of a shape-changing materialand a smart material; and configuring the optical imaging system toautomatically autofocus to a predetermined point relative to a medicalinstrument only after a time lag, whereby erroneous focus change isavoided.
 19. A method of autofocusing using an intraoperative opticalimaging system during a medical procedure, the method comprising:providing an intraoperative optical imaging system for imaging a targetduring a medical procedure, providing the intraoperative optical imagingsystem comprising: providing an optical assembly, providing the opticalassembly comprising providing at least one moveable zoom optic andproviding at least one moveable focus optic and having an aperture;providing a zoom actuator for positioning the at least one moveable zoomoptic; providing a focus actuator for positioning the at least onemoveable focus optic; providing a controller for independentlycontrolling the at least one moveable zoom optic and the at least onemoveable focus optic by respectively using the zoom actuator and thefocus actuator in response to received control input; and providing acamera for capturing an image of the target from the optical assembly,wherein providing the zoom optics providing the focus optics comprisesrespectively providing each of the at least one moveable zoom optic andthe at least one focus optic with at least one independently moveablelens operably coupled with a linear stage and a conveyer system operablycoupled with the zoom actuator and the focus actuator for respectivelycontrolling positioning the at least one moveable zoom optic and the atleast one focus optic, the linear stage comprising a linear guide and aguide actuator, and the conveyer system comprising a conveyor beltmechanism, and the conveyer system operably coupled with the zoomactuator and the focus actuator for respectively controlling positioningthe at least one moveable zoom optic and the at least one focus optic,and wherein providing at least one of the zoom actuator and the focusactuator comprises providing at least one of a shape-changing materialand a smart material; and configuring the optical imaging system toautomatically autofocus to a predetermined point relative to a medicalinstrument only after a time lag, whereby erroneous focus change isavoided; determining a working distance between an imaging target andthe aperture; determining a desired position of the focus optics basedon the working distance; and positioning the focus optics at the desiredposition.
 20. The method of claim 19, further comprising: receivinginformation about position and orientation of the optical imagingsystem; using the received information to determine the working distancerelative to the imaging target; determining there is no medicalinstrument with a focus point within a current field of view of theoptical imaging system; performing autofocusing on another imagingtarget different other than the focus point; receiving information aboutposition and orientation of the medical instrument; determining positionof the focus point based on the position and orientation of the medicalinstrument; determining the working distance using the determinedposition of the focus point; receiving information about position andorientation of the positioning system; determining position andorientation of the optical imaging system using known position andorientation of the optical imaging system relative to the positioningsystem; and using the determined position and orientation of the opticalimaging system to determine the working distance, wherein theinformation about the position and orientation of the medical instrumentis received from an external navigation system, wherein the informationabout position and orientation of the optical imaging system is receivedfrom an external navigation system, wherein the optical imaging systemis mounted on a positioning system, wherein the information aboutposition and orientation of the positioning system is received from anexternal navigation system, and wherein the imaging target is a focuspoint defined relative to a medical instrument.